refactor: code block & math block syntax highlighting#2873
Conversation
De-duplication: when an extension with a given key is already registered, skip registering another one with the same key (the first registration wins). This lets an extension declare a dependency on another via `blockNoteExtensions` without conflicting when that same extension is also registered directly by the user. Ordering: a sub-extension declared via `blockNoteExtensions` is a dependency of the extension that declares it, so it now runs before its parent. The dependency is recorded as the sub is resolved (before the de-duplication check), so it holds even when multiple parents declare the same sub-extension and when a parent has a higher base priority via `runsBefore`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… & math block can configure a highlighter for
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Looks good, I agree that trying to merge duplicate extensions is probably more effort than what we need atm and better to drop them.
Though I'm not a bit wary of keeping the highlighting option in meta when there is a specific extension for it. I think we should be careful when adding things to the custom block API that we don't end up with like a dozen unrelated config options that all live under meta. Probably fine in this case but yeah it's something we should be careful with. I guess an alternative would be to not bundle the syntax highlight extension with the code or math blocks, and make it completely standalone + configure highlighting there.
|
My thinking there is that it is inflexible (and annoying) to have a shared dep have to be aware of the details of each block and it's special cases. I would rather that the blocks could declare how they should be treated by an extension instead of the extension having to be aware of the block. The reason is two fold:
A perfect example of this is the slash menu. Right now, we have the default blocks automatically injected into the shared slash menu array. When, if we did it in this way, we'd have each block declaring what sort of block it is and how to show itself in the slash menu. So, the slash menu would be simpler and including new things into the slash menu would be easier. Rather than having to modify the shared state somewhere else. |
YousefED
left a comment
There was a problem hiding this comment.
The trouble is how to handle both code block & the math extension wanting to configure the highlighter in different ways. Well, extensions can now only be registered once per key with #2872 so it will just silently drop one of the configurations and use one of the highlight instances to try to highlight both the math & code blocks.
I think this is a fine tradeoff for now, since you can really just leave the code block syntax highlighting on & get both code blocks & math blocks "for free". There is a larger question on how to do this in the more general case for extensions which depend on each other, but that is just a hard problem in programming in general.
I'm curious if we can come up with a cleaner solution. Looking at the code, can't we:
- still make sure there's only 1
createHighlightPlugin(the shiki prosemirror extension) - but, extend the
lazyParserto pick the right highlighting data (which could come from different configurations)?
|
Alright we are going to go a different direction here. Since syntax-highlighting is an optional thing, we will make it a separate extension that needs to be configured with a highlighter. We'll remove the createHighlighter config on code-block and make the user provider a SyntaxHightlighter extension. If this extension is present then math block and code block can get highlighted on. For something like the Users (separate) which is needed by both Comments & Suggestions extensions, we won't make it an extension just an instance of an object which will need to be configured for each (and required if required), withCollaboration would make this easy. So, this is closer to what Matthew had to start, just as an optional extension rather than as an editor options |
…n duplicate keys
Syntax highlighting is now a standalone extension the user adds to the
editor's `extensions`, rather than config wired into each block. Removes
`createHighlighter` from `CodeBlockOptions` and the math block options, and
removes the dead editor-level `syntaxHighlighting` option. Blocks keep
declaring their language via `meta.highlight`, so the extension highlights
whichever blocks are present when it's added; when it's absent, they render
as plain text.
`@blocknote/code-block` now exports a pre-configured `syntaxHighlighter`
extension (bundled Shiki + github themes); custom highlighters use the
`SyntaxHighlightingExtension` factory from `@blocknote/core`.
Also makes the ExtensionManager throw on any duplicate extension key instead
of silently de-duplicating ("first wins"). A shared extension must be
registered once rather than by each consumer. This is safe because the only
real duplicate-key case (code + math both registering the highlighter) is
removed by this refactor; the sole `blockNoteExtensions` user
(CollaborationExtension) has unique sub-extension keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integrates the code/math block preview refactor (source-with-preview popups, inline math, diagram block) with the user-provided syntax-highlighter work. Conflict resolution: - Code block: dropped the highlighter self-registration (already removed on both sides); kept previews' code-block extension list. - Math block: took previews' restructure (block/ + inlineContent/ dirs; removed the flat block.ts/reactBlock.tsx) and re-added `highlight: () => "latex"` to the new createReactMathBlockSpec meta so the syntax highlighter still highlights math sources. - math-block example: took previews' math + inline-math version, swapped the removed editor-level `syntaxHighlighting` option for `extensions: [syntaxHighlighter]`. - ExtensionManager: kept the throw-on-duplicate-key behavior alongside previews' inline-content extension registration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…highlighting Refactor SourceBlockWithPreviewExtension and its inline sibling into single editor-wide extensions, mirroring SyntaxHighlightingExtension: - Register both once as default extensions instead of per block/inline spec. - Blocks/inline content opt in via a `meta.hasPreview` flag rather than the extension taking a `blockType`/`inlineContentType`; drop `key`, `runsBefore`, the always-true `hasPreview` callback, and the `enterBehaviour` option. - Multi-line preview blocks (diagram) declare `meta.hardBreakShortcut: "enter"`; the extension reads it from the spec to decide Enter behavior. Note that the shared KeyboardShortcutsExtension reads hardBreakShortcut off the block config, which carries no meta - flagged with a comment for a follow-up. Enable syntax highlighting for inline content (e.g. inline math): - collectHighlightNodeTypes now also collects inline-content specs with a `meta.highlight`; add `highlight`/`hasPreview` to the inline meta types. - Inline math declares `highlight: () => "latex"`. - Bump prosemirror-highlight to 0.15.3, which collects nodes by `node.inlineContent` (ocavue/prosemirror-highlight#137) so inline nodes are highlighted alongside text blocks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| // Diagram source is Mermaid, so highlight it as such when the syntax | ||
| // highlighting extension is present. | ||
| // | ||
| // NOTE: this currently has no visible effect. Shiki's Mermaid grammar is | ||
| // a Markdown injection (`injectionSelector: "L:text.html.markdown"`) that | ||
| // only tokenizes inside a ```` ```mermaid ```` fence, so it produces no | ||
| // tokens for bare Mermaid source. See | ||
| // https://github.com/shikijs/shiki/issues/973 - once that's resolved | ||
| // upstream, highlighting should start working with no change here. | ||
| highlight: () => "mermaid", |
There was a problem hiding this comment.
So, I figured out that the highlighting was working, it just only highlights something as ```mermaid which is an annoying limitation. I bumped an issue, but I'm not sure how easy it is to fix or not: shikijs/shiki#973
| "fast-deep-equal": "^3.1.3", | ||
| "lib0": "^0.2.99", | ||
| "prosemirror-highlight": "^0.15.1", | ||
| "prosemirror-highlight": "^0.15.3", |
There was a problem hiding this comment.
I figured out why math inline nodes weren't being highlighted & made a PR upstream to fix it & this is the result of that: ocavue/prosemirror-highlight#137
|
So, the new approach is:
I want to keep moving forward with this pattern of stashing things relevant to extensions in spec.meta because it cuts down on the number of special casing we need to have for blocks, an extension should just always work whether it knows about the blocks you have or not. |
Summary
Two related refactors that make editor-wide behaviors (syntax highlighting, source-with-preview) into single user- or default-provided extensions instead of configuration wired into each block, and — on the back of the highlighting change — extend syntax highlighting to inline content (e.g. inline math).
1. Syntax highlighting is a standalone extension
Syntax highlighting is now a standalone extension the user provides to the editor, rather than configuration wired into each block.
Previously the highlighter (
createHighlighter) lived on the code block's options (and the math block's), and each block self-registered a sharedSyntaxHighlightingExtensionsingleton. That meant the highlighter config was duplicated onto every block that wanted it, and — relying on silent de-dup-by-key — whichever block registered first silently won the config while the other was dropped.Now:
extensionsoption. The block schema is unchanged — code and math blocks keep declaring how to highlight their contents viameta.highlight((block) => block.props.languagefor code,() => "latex"for math). When the extension is present, it highlights whichever blocks declare a language; when it's absent, they render as editable plain text.@blocknote/code-blockexports a ready-to-usesyntaxHighlighterextension, pre-configured with its bundled Shiki highlighter (github themes). For a custom highlighter, build one with theSyntaxHighlightingExtensionfactory from@blocknote/core.Rationale
The earlier approach depended on the ExtensionManager silently de-duplicating extensions by key ("first registration wins"). That resolves the "code block and math block both configure a highlighter" conflict, but does so by silently discarding one configuration — a subtle bug waiting to happen, and it leaves it ambiguous which instance's config is actually used.
Making syntax highlighting a single user-provided extension removes the conflict at its source: there's exactly one highlighter, registered once, and it highlights every block that declares a language. Config is no longer duplicated across blocks.
On the back of that, this PR also replaces silent de-duplication with a hard error: registering two extensions with the same key now throws, from any source (direct registration or a
blockNoteExtensionsdependency). Extension keys must be unique; a shared extension must be registered once rather than by each consumer. This is safe because the only real duplicate-key case in the codebase was the code + math highlighter, which this refactor removes — and the soleblockNoteExtensionsuser (CollaborationExtension) has unique sub-extension keys, verified at runtime.2. Inline content syntax highlighting
Syntax highlighting now also works for inline content (e.g. inline math), not just blocks.
SyntaxHighlightingExtensiondiscovers highlightable nodes by scanning bothblockSpecsandinlineContentSpecsfor ameta.highlightcallback (blocks withcontent: "inline", inline content withcontent: "styled"). Inline math declareshighlight: () => "latex", so its LaTeX source is highlighted in the editable popup like the block version.prosemirror-highlight: it previously collected only text-block nodes, so inline nodes were never highlighted. Since v0.15.3 it collects nodes bynode.inlineContent(ocavue/prosemirror-highlight#137), so inline nodes that hold inline content are visited alongside text blocks. This PR bumpsprosemirror-highlightto^0.15.3; no local patch is needed.3. SourceBlockWithPreview is a single default extension
SourceBlockWithPreviewExtension(and its inline siblingSourceInlineContentWithPreviewExtension) are now single editor-wide extensions registered once by default, mirroring the syntax-highlighting pattern — instead of being instantiated per block/inline-content spec.Previously each preview block constructed the extension itself, passing a bespoke
key, ablockType/inlineContentType, an always-() => truehasPreviewcallback, anenterBehaviourstring, andrunsBefore.Now:
getDefaultExtensionsand registered once. Blocks/inline content opt in by declaringmeta.hasPreview: truein their spec (self-discovered from the schema, the same waymeta.highlightis), instead of the extension being told a block/inline type.hasPreviewcallback,key, andrunsBeforeoptions are gone.enterBehaviour: "close" | "newline"option is replaced by the existing, schema-drivenmeta.hardBreakShortcut: single-line sources (math) leave it default; multi-line sources (diagram) sethardBreakShortcut: "enter", which the preview extension reads from the spec to decide whether Enter inserts a line break or commits/closes the popup.Changes
@blocknote/corecreateHighlighterfromCodeBlockOptions; remove the code block's self-registration of the highlighting extension (keepsmeta.highlight). Remove the dead editor-levelsyntaxHighlightingoption fromBlockNoteEditorOptions.ExtensionManagernow throws on any duplicate extension key instead of silently de-duplicating; theblockNoteExtensionsordering edge is retained. Fix stale doc comments to describe the realmeta.highlightmechanism.collectHighlightNodeTypesalso collects inline-content specs with ameta.highlight; addhighlight/hasPreviewto the inline meta types. Bumpprosemirror-highlightto^0.15.3.SourceBlockWithPreviewExtensionandSourceInlineContentWithPreviewExtensioninto editor-wide singletons registered ingetDefaultExtensions; drop thekey/blockType/inlineContentType/hasPreview-callback/enterBehaviour/runsBeforeoptions in favor ofmeta.hasPreview+meta.hardBreakShortcut. AddhasPreviewtoBlockConfigMetaand the inline meta types. Note thatKeyboardShortcutsExtensionreadshardBreakShortcutoff the block config (which carries nometa) — flagged with a comment for a follow-up; the preview extension reads it from the spec implementation instead.@blocknote/code-block: dropcreateHighlighterfromcodeBlockOptions; export a pre-configuredsyntaxHighlighterextension.@blocknote/math-block: remove the highlighter option and self-registration; keephighlight: () => "latex"(block and inline); declaremeta.hasPreviewinstead of constructing the preview extension.@blocknote/diagram-block: declaremeta.hasPreview+meta.hardBreakShortcut: "enter"+highlight: () => "mermaid"instead of constructing the preview extension withenterBehaviour: "newline".theming/code-block,theming/custom-code-block(bring-your-own bundle),custom-schema/math-block, andcustom-schema/diagram-blocknow pass the extension viaextensions: [...]; code-blocks docs updated to the new API.Impact
Breaking changes to the in-progress branch API:
createHighlighteris no longer accepted oncreateCodeBlockSpec/createMathBlockSpecoptions, and the editor-levelsyntaxHighlightingoption is removed. To enable highlighting, addsyntaxHighlighter(or your ownSyntaxHighlightingExtension) toextensions.SourceBlockWithPreviewExtension/SourceInlineContentWithPreviewExtensionno longer take options (key,blockType/inlineContentType,hasPreview,enterBehaviour,runsBefore). Preview blocks opt in viameta.hasPreview(andmeta.hardBreakShortcutfor multi-line source behavior) rather than constructing the extension.Testing
ExtensionManager(unique-key throws for direct dupes,blockNoteExtensionsdupes, and shared-sub dupes; single dependencies still register),code-block,SyntaxHighlighting(includingcollectHighlightNodeTypesblock/inline collection), a newinlineHighlightsuite (inline nodes are highlighted, non-matching and atom inline nodes are not), andmath-block/diagram-blockpopup keyboard handling (incl. math single-line commit vs diagram multi-line newline). Full@blocknote/coresuite passes.tsgo) clean across core, code-block, math-block, diagram-block, react, and the modified examples. Lint clean.theming/code-block,theming/custom-code-block,custom-schema/math-block, andcustom-schema/diagram-blockall produce Shiki-highlighted token spans — including inline math, whose LaTeX source now shows highlighted tokens (e.g.\picolored) where it previously did not. Verified the collaboration examples and the full default-block schema mount with no duplicate-key errors.Checklist